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L REAL PARTIES IN INTEREST 



The real parties in interest are SARNOFF CORPORATION (Assignee) and AUTOLIV 
ELECTRONICS by virtue of an assignment executed by the inventors (Appellants) to 
SARNOFF CORPORATION AND AUTOLIV ELECTRONIS (recorded by the Assignment 
Branch of the U.S. Patent and Trademark Office on January 29, 2004 at Reel/Frame 
014944/0602. 

11. RELATED APPEALS AND INTERFERENCES 

None. 

III. STATUS OF CLAIMS 

In the application under appeal, claims 1-28 are pending. Claims 1-5, 7, 13-14, 16 and 
20-22 stand rejected under 35 U.S.C. § 102(b) as being anticipated by Awe Franke et al, 
Autonomous Driving Goes Downtown , I.E.E.E. Intelligent Systems, pages: 40-48, 1998 
(hereinafter "Franke"). Claims 6, 15 and 23 stand rejected as being unpatentable over Franke in 
view of Ming Yang et al.. Vision-based Real-time Obstacles Detection and Tracking for 
Autonomous Vehicle Guidance , Real-time Imaging VI, Proceedings of SPIE Vol. 4666, pages 
65-74, 2002 (hereinafter 'Tang"). Claims 8-12, 17-19, 24-26, and 27-28 are objected to as being 
dependent upon a rejected based claim, but have been deemed allowable if rewritten in 
independent form including all the limitations of the base claim and any intervening claims. The 
rejection of claims 1-7, 13-16 and 23 are appealed. 
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IV. STATUS OF AMENDMENTS 



Appellants have submitted no amendments after the non-fmal rejection. All amendments 
prior to the close of prosecution on the merits have been entered. 



V. SUMMARY OF CLAIMED SUBJECT MATTER 



Independent claim 1 recites a method of detecting an imminent collision. The method 
includes capturing and preprocessing imagery of a scene proximate a platform. Capturing and 
preprocessing imagery of a scene proximate a platform is described in the specification as filed 
at least at paragraph 24 and in Figure 3. The method further includes producing fi-om the 
imagery a depth map. Producing a depth map from the imagery is described in the specification 
as filed at least at paragraph 24 and in Figure 3. The method further includes tessellating the 
depth map into a number of patches and selecting a plurality of the patches of the depth map for 
processing. Tessellating the depth m^ into a number of patches is described in the specification 
as filed at least at paragraph 30 and in Figure 6 and selecting a plurality of the patches of the 
depth map for processing is described in the specification as filed at least at paragraph 32 and in 
Figure 7. The processing includes classifying the selected plurality of patches of the depth map 
into a plurality of classes. Processing including classifying the selected plurality of patches of 
the depth map into a plurality of classes is described in the specification as filed at least at 
paragraph 30 and in Figure 6. The method fiirther includes detecting a potential threat in the 
tessellated depth map during the processing of the selected plurality of the patches. Detecting a 
potential threat in the tessellated depth map during the processing of the selected plurality of the 
patches is described in the specification as filed at least at paragraphs 30 and 44 and in Figure 7. 
The method further includes estimating the size of the detected potential threat. Estunating the 
size of the detected potential threat is described in the specification as filed at least at paragraphs 
26 and 45 and in Figure 4. The method further includes estimating the position of the detected 
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potential threat. Estimating the position of the detected potential threat is described in the 
specification as filed at least at paragraphs 26 and 45 and in Figure 4. The method further 
includes estimating the velocity of the detected potential threat. Estimating the velocity of the 
detected potential threat is described in paragraphs 26 and 46 and in Figure 4. The method 
further includes performing a trajectory analysis of the detected potential threat using the 
estimated position and the estimated velocity. Performing a trajectory analysis of the detected 
potential threat is described in the specification as filed at least at paragraph 28 and in Figure 4. 
The method further includes performing a collision prediction based on the trajectory analysis. 
Performing a collision prediction based on the trajectory analysis is described in the specification 
as filed at least at paragraph 28 and in Figure 4. 

Independent claim 13 recites a coUision detection system. The system includes a stereo 
camera pair for providing imagery of a scene proximate a platform. The stereo camera pair for 
providing imagery of a scene proximate a platform is described in the specification at least at 
paragraphs 22 and 24 and in Figures 2 and 3, references 200 and 202. The system fiirther 
includes a stereo image preprocessor for preprocessing the imagery. The stereo image 
preprocessor for preprocessing the imagery is described at least at paragraph 24 and in Figure 3, 
reference 300. The system further includes a depth map generator for producing a depth map 
&om the preprocessed imagery. The depth map generator is described in the specification at 
least at paragraph 24 and in Figure 3, reference 302. The system fiirther includes a colhsion 
detector for tessellating the depth map into a number of patches, selecting a plurality of the 
patches of the depth map for processing. The collision detector for tessellating a depth map into 
a number of patches, selecting a plurality of the patches of the depth map for processing is 
described in the specification at least at paragraphs 24, 30 and 32 and in Figure 3, reference 304. 
The processing includes classifying the selected plurality of patches of the depth map into a 
plurality of classes. The processing including classifying the selected plurality of patches of the 
depth map into a plurality of classes is described in the specification at least at paragraph 30 and 
in Figure 6. The processing also includes detecting a potential threat in the tessellated depth map 



during the processing of the selected plurality of the patches. The processing including detecting 
a potential threat in the tessellated depth map during the processing of the selected pluraHty of 
the patches is described in the specification at least at paragraphs 30 and 44 and in Figure 7. 
The collision detector estimates size, position, and velocity of the detected potential threat. The 
collision detector performs a trajectory analysis of the detected potential threat using the 
estimated position and the estimated velocity. The collision detector predicts a collision based 
on the trajectory analysis. The collision detector determines if a collision is imminent based on 
the collision prediction and on the estimated size. The collision detector that estimates size, 
position, and velocity of the detected potential threat, performs a trajectory analysis of the 
detected potential threat using the estimated position and the estimated velocity, predicts a 
collision based on the trajectory analysis and determines if a collision is imminent based on the 
collision prediction and on the estimated size is described in the specification at least at Abstract 
and paragraphs 20, 25, 26, 28 and 46 and in Figure 4. 

Independent claim 20 recites a computer readable medium having stored thereon a 
plurality of instructions, the plurality of instruction including instructions which, when executed 
by a processor causes the processor to perform steps. The computer readable medium having 
stored thereon a plurality of instructions, the plurality of instruction instructions which, when 
executed by a processor causes the processor to perform steps is described in the specification as 
filed at least at paragraph 23 and in Figures 1 and 2, reference 108. The steps include capturing 
and preprocessing an imagery of a scene proximate a platform. C^turing and preprocessing an 
imagery of a scene proximate a platform is described in the specification as filed at least at 
paragraph 24 and in Figure 3. The steps further include producing from the imagery a depth 
map. Producing from the imagery a depth map is described in the specification as filed at least at 
paragraph 24 and in Figure 3. The steps fiirther include tessellating the depth map into a number 
of patches. Tessellating the depth map into a number of patches is described in the specification 
as filed at least at paragraph 30 and in Figure 6. The steps fiurther include selecting a plurality of 
the patches of the depth map for processing. Selecting a plijraUty of the patches of the depth 
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map for processing is described in the specification as filed at least at paragraph 32 and in Figure 
7. The processing includes classifying the selected plurality of patches of the depth map into a 
plurality of classes. Processing including classifying the selected plurality of patches of the 
depth map into a plurality of classes is described in the specification as filed at least at paragraph 
30 and in Figure 6. The steps further include detecting a potential threat in the tessellated depth 
map during the processing of the selected plurality of the patches. Detecting a potential threat in 
the tessellated depth map during the processing of the selected plurality of the patches is 
described in the specification as filed at least at paragraphs 30 and 44 and in Figure 7. The steps 
fiirther include estimating the size of the detected potential threat. Estimating the size of the 
detected potential threat is described in the specification as filed at least at paragraphs 26 and 45 
and in Figure 4. The steps further include estimating the position of the detected potential threat. 
Estimating the position of the detected potential threat is described in the specification as filed at 
least at paragraphs 26 and 45 and in Figure 4. The steps further include estimating the velocity 
of the detected potential threat. Estimating the velocity of the detected potential threat is 
described in the specification as filed at least at paragraphs 26 and 46 and in Figure 4. The steps 
further include performing a trajectory analysis of the detected potential threat using the 
estimated position and the estimated velocity. Performing a trajectory analysis of the detected 
potential threat using the estimated position and the estimated velocity is described in the 
specification as filed at least at paragraph 28 and in Figure 4. The steps further include 
performing a collision prediction based on the trajectory analysis. Performing a collision 
prediction based on the trajectory analysis is described in the specification as filed at least at 
paragraph 28 and in Figure 4. 

References in this brief to supporting portions of the specification and drawings are given 
to provide exemplary embodiments, not to provide limitations to the claims. 
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VI. GROUNDS OF REJECTION TO BE REVIEWTD ON APPEAL 



Appellants respectfully request that the Board of Patent Appeals and Interferences review 
the following grounds of rejection on appeal: 

1. Whether claims 1-5, 7, 13-14, 16, and 20-22 are patentable under 35 U.S.C. § 
102(b) over France 

2. Whether claims 6, 15 and 23 are patentable under 35 U.S.C. § 103(a) over Franke 
in view of Yang. 

VII. ARGUMENT 

Appellants respectfully submit that claims 1-5, 7, 13-14, 16, and 20-22 are patentable 
over the prior art of record. 

Claims 1-S. 7. 13-14. 16 and 20-22 Are Patentable over Franke 

Independent claims 1, 13 and 20 are rejected under 35 U.S.C. § 102(b) as being 
anticipated by Franke. Appellants respectfully disagree. 

Appellants assert that independent claims 1, 13 and 20 are patentable over Franke 
because Appellants contend that Franke discloses a different method, system and computer 
readable medium, respectively, from what is claimed in claims 1,13 and 20. Appellants further 
contend that the Examiner, when rejecting claims 1, 13 and 20 under 35 U.S.C. § 102(b), failed 
to give weight to express limitations of the claims that are not disclosed, taught or suggested by 
Franke. "A prior art reference anticipates a claim only if the reference discloses, either expressly 
or inherently, every limitation of the claim.... Absence from the reference of any claimed 
element negates anticipation." Rowe v. Dror. 112 F3.3d 473, 478, 42 USPQ2d 1550 Fed. Cir. 
(1997) (internal citations omitted). Franke does not disclose, teach or suggest all the limitations 
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of claim 1. In general. Appellants contend that Franke does not disclose a method of detecting 
an imminent colUsion claimed in claim 1. Illustrative of this is that Franke does not disclose the 
method of detecting an imminent collision that includes classifying a selected plurality of 
patches of the depth map into a plurality of classes as claimed in claim 1 . 

Firstly, Franke does not teach classifying patches of a depth map . As is known to those 
of skill in the art, a depth map is a 2-dimensional array of values (sometimes also called a depth 
image, range map or range image), each point of which represents the depth to a point in a scene. 
Also, as claimed, the depth map is tessellated into a plurality of patches, and the patches are then 
classified. As is known to those of skill in the art, to tessellate is to form of small squares or 
blocks, as floors or pavements or to form or arrange in a checkered or mosaic pattern. Franke 
does not describe classifying portions of a depth map, but instead, describes classifying pixels of 
an image . 

Secondly, as claimed in the present application, a selected number of the patches of a 
depth map are classified into a plurality of classes. Franke does not describe classifying selected 
portions, but instead teaches classifying a whole image. 

Thus, Appellants assert that Franke fails to disclose all the elements of claim 1 and 
Appellants further assert that the Examiner failed to properly consider this claim limitation in his 
rejection. For example; 

Classifying a plurality of patches of the depth map: 

The Examiner contends that Franke discloses classifying a plurality of patches of a depth 
map into a plurality of classes. For this, the Examiner points to Franke, which describes that a 
system can "detect and classify different additional traffic participants, such as bicyclists or 
pedestrians" {Franke at page 41, column 1), that a "polynomial classifier subsequently classifies 
detected lane boundaries such as curbs, markings or [a] clutter" {Franke at page 43, column 1), 
and "[t]he classifications stages [involve] color, shape and pixel values" {Franke at page 44, 
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column 2). In none of these sections, however, does Franke disclose classifying patches of a 
depth map into a plurality of classes as will be described in greater detail below. 

When Franke describes a system to "detect and classify different additional traffic 
participants, such as bicyclists or pedestrians;" Franke is describing tasks to be performed to 
build a system without any teaching or suggestion of how to implement this task. This portion of 
Franke does not describe classifying patches of a depth map. When Franke describes that a 
"polynomial classifier subsequently classifies detected lane boundaries such as curbs, markings 
or clutter," Franke is simply disclosuig using a polynomial method to classify the features 
detected on a road which includes identifying the target types in the region of interest and 
rejecting any clutter and background. As known to those of skill in the art, a polynomial 
classifier is defined as a method of grouping objects into one or more groups, or classes, based 
on the best-fit of a parameterized polynomial equation to the observed data. Again, this portion 
of Franke involves classifying polynomials of an image, not a depth map. Clearly, the 
polynomial classifier as disclosed in Franke fails to teach or suggest classifying patches of a 
depth map into a plurality of classes. Franke also describes that "classification, is done with an 
RBF classifier in [a] multi stage process," and that "a color-normalized pictograph [is] extracted 
firom the original image," wherein the "classifications stages involve color, shape and pixel 
values." As known to those of skill in the art, a RBF classifier is a radial basis function classifier 
which is defined as a real-valued function whose value depends on the distance firom an origin. 
The RBF classifier as disclosed in Franke classifies a pictograph firom an original image. So, 
again, Franke describes classifying a pictograph from an image, but not classifying patches of a 
depth map. Thus, Franke fails to teach or suggest any type of classifying of the depth map 
including classifying patches of a depth map. Therefore, the cited portions of Franke cannot 
support the Examiner's rejection of claim 1. 

On the contrary, as discussed above, Franke does not describe classifying patches of a 
depth map, but instead teaches a system in which classification is performed on an entire image. 
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Note that on page 41, column 2, Franke teaches that a "feature-based approach classifies each 
pixel according to gray values of its four direct neighbors." Thus, Franke teaches classification 
of an entire image of the scene (figure 1 of Franke) by classifying each pixel in the image. 
Whereas, in the present application, a plurality of patches of the depth map are classified, as 
recited in amended independent claim 1, 13 and 20. Further, these images ot Franke are stereo 
images and not a depth map as claimed in the present invention. See page 41, column 2 and 
Figure 1 of Franke. As is known to those of skill in the art, a stereo image is a pair of images 
taken of a single scene by two cameras, whose relative orientation is typically pre-determined by 
calibration. One skilled in the art understands that the stereo image is not same as the depth map, 
which, as discussed above, is a 2-dimensional array of values (sometimes also called a depth 
image, range map or range image), each point of which represents to a point in a scene. While 
Franke does separately describe a depth map, Franke does not describe classifying patches of a 
depth map. Instead, after classification of the stereo image, Franke describes extracting 
structures from the image to recognize road boundaries and then removing all features on the 
road plane to generate a 2-dimensional depth map containing the remaining features including 
the objects that are subsequently tracked. (See page 41, columns 2 and 3 and page 42, column 1 
of Franke). Thus, Franke describes classifying an image, removing objects from the image, and 
then creating a depth map, but Franke does not describe classifying portions of a depth map. 
Clearly, Franke does not describe any classifying of the depth map, let alone classifying a 
plurality of patches of the depth map . 

Classifying a selected plurality of patches of the depth map into a plurality of classes: 

In the present ^plication, only parts (selected patches) of the depth map are classified, 
not an entire depth m^. Support for this is provided in paragraphs 30 and 32 and Figures 6 and 
7 of the present application. On the contrary, Franke teaches a system in which classification is 
performed on an entire image. Note that on page 41, column 2, Franke teaches that a "feature- 
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based approach classifies each pixel according to gray values of its four direct neighbors." Thus, 
Franke teaches classification of an entire image of the scene (figure 1 of Franke) by classifying 
each pixel in the image. Whereas, in the present application, a selected plurality of patches of 
the depth map are classified, as recited in independent claims 1,13 and 20. Thus, as claimed in 
the present application, only selected parts of a depth are classified. Support for this is 
provided in paragraphs 30 and 32 and Figures 6 and 7 of the present application. For example, at 
step 706 in Figure 7, a determination is made as to whether a patch is dense enough to be used 
for processing and only those patches with sufficient density are selected for processing. 

Therefore, the cited disclosure oi Franke fails to meet the MPEP's requirement that "[a] 
claim is anticipated only if each and every element as set forth in the claim is foimd, either 
expressly or inherently described, in a single prior art reference." Verdeeaal Bros.. Inc. v. Union 
Oil Co. . 814 F.2d 628, 631, 2 USPQ2d 1051, 1053 (Fed. Cir. 1987) (Emphasis Added). 

In the system and method of the present invention as claimed, the depth map is first 
produced, which is tessellated into patches and then further classified to detect a potential threat. 
In contrast, Franke describes an apphcation that uses a depth map and, also describes a separate 
and distinct application that employs object classification. Thus, Franke describes two separate 
applications, which are independent of each other. First is the stereo-based obstacle detection 
and tracking application and second is the object recognition application. With regard to the 
first system, on page 40, column 2, Franke describes a system that "includes stereo vision for 
depth-based obstacle detection and tracking and a framework for monocular detection and 
recognition of relevant objects." Also, on page 46, colimm 2 to column 3, Franke recites two 
methods of detection as "[c]olor clustering on monocular images in a combined color and 
position feature space" and "3D segmentation on stereo vision." As is known to those of skill in 
the art, a monocular image is one image taken of a single scene by a single camera. This 
detection of the monocular image does not provide the depth perception of the image. One 
skilled in the art understands that a monocular image is not same as the stereo image, which, as 
discussed above, is a pair of images taken of a single scene by two cameras, whose relative 
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orientation is typically pre-determined by calibration. This clearly indicates that the object 
recognition of Franke does not utilize the stereo depth map of obstacle detection since it is 
monocular. So, the steps of object recognition do not occur subsequent to the disparity image 
created for obstacle detection and tracking. Therefore, while Franke does describe both object 
detection and the use of a depth map, these features are not described as being part of the same 
system, but instead, are described as being separate and distinct. Thus, the depth map described 
by Franke is not "arranged as in that claim" as recited by claim 1 of the present application. 
Furthermore, "Anticipation under Section 102 requires the presence in a single prior art 
disclosure of all elements of a claimed invention arranged as in that claim" quoting Connell vs. 
Sears. Roebuck & Co . 772 F.2d 1542, 1548, 220 USPQ 193, 198 (Fed. Cir. 1983). 

The above discussion shows features of claim 1 are not disclosed or suggested hy Franke 
Thus, for an^r one of the several reasons discussed above, it is believed that Franke fails to 
anticipate the invention of claim 1 under 35 USC § 102(b), and that the rejection should be 
reversed. Such action is respectMly requested. Appellants further believe that the rejection of 
dependent claims 2-5 should be reversed at least because such clauns depend directly or 
indirectly from such parent claim 1, which is believed patentable. 

Independent claims 13 and 20 were rejected based on an application o^ Franke in a way 
that is believed incorrect as indicated above with respect to claim 1. Thus, at least for the 
reasons set forth above, Appellants believe that the rejection of such claims should be reversed. 
Appellants further beheve that the rejection of dependent claims 14, 16-19 and 21-22 should be 
reversed at least because such claims depend directly or indirectly ftom such parent claims 13 
and 20, which are believed patentable. 
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Claims 6, 15 and 23 Are Patentable Over Franke and Yan2 

Claims 6, 15 and 23 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Franke as applied to claims 1, 13 and 20 above, and ftirther in view of Yang. Appellants 
respectfully disagree. 

The rejection of such claims was made based on an ^phcation of Franke, plus an 
additional application of Yang for elements that the Examiner recognizes are not disclosed by 
Franke. Appellants beUeve that the application of Franke to such claims was made in a way that 
is incorrect as indicated above with respect to claims 1, 13 and 20. Thus, there are still elements 
of the claims that are not taught or suggested by the references, even if Franke were to be 
combined with Yang. Therefore, it is believed that the rejection of such claims 6, 15 and 23 
under 35 U.S.C. § 103(a) should be reversed, and such action is respectfiiUy requested. 

CONCLUSION 

For the reasons stated above, claims 1-7, 13-16 and 20-23 are patentable over the prior art 
of record, and the rejections of those claims under 35 U.S.C. § 102(b) and 35 U.S.C. § 103(a) are 
improper and should be withdrawn. Appellants respectfully ask the Board to reverse the 
Examiner's rejections with instructions to allow the claims. 

The USPTO is directed and authorized to charge all requuxd fees to Deposit Account No. 
501358. Appellant's undersigned agent may be reached by telephone at (973) 597-2500. All 
correspondence should continue to be directed to our address listed below, 

/Rohini K. Garg/ 
Rohini K. Garg 
Registration No. 45,272 
Registered Patent Agent 

DOCKET ADMINISTRATOR 
LOWENSTEIN SANDLER PC 
65 Livingston Avenue 
Roseland,NJ 07068 
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VIII. CLAIMS APPENDIX 



1. (Previously Presented ) A method of detecting an imminent collision comprising the steps of: 

capturing and preprocessing imagery of a scene proximate a platform; 
producing from the imagery a depth m^; 

tessellating the depth map into a number of patches and selecting a plurality of the 
patches of the depth map for processing, wherein said processing comprise classifying the 
selected plurality of patches of the depth map into a plurality of classes; 

detecting a potential threat in the tessellated depth map during the processing of the 
selected plurality of the patches; 

estimating the size of the detected potential threat; 

estimating the position of the detected potential threat; 

estimating the velocity of the detected potential threat; 

performing a trajectory analysis of the detected potential threat using the estimated 
position and the estimated velocity; and 

performing a collision prediction based on the trajectory analysis. 

2. (Original) The method of claim 1, further including determining if a collision is imminent 
based on the collision prediction and on the estimated size of the potential threat. 

3. (Original) The method of claim 1, further including filtering the estimated position and 
filtering the estimated velocity before performing trajectory analysis. 

4. (Previously Presented) The method of claim 3 wherein the filtering includes Kahnan 
filtering. 

5. (Original) The method of claim 1 wherein estimating the velocity of the detected potential 
threat includes the step of identifying 2-dimensional feature correspondences from imagery 
produced in different time frames. 
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6. (Previously Presented) The method of claim 5 wherein estimating the velocity of the detected 
potential threat further includes the step of obtaining 3D correspondences from the 2- 
dimensional feature correspondences and from the depth map. 

7. (Original) The method of claim 6 wherein estimating the velocity of the detected potential 
threat further includes the step of estimating velocity using Random Sample Consensus. 

8. (Previously Presented) The method of claim 1 wherein said processing the selected plurality 
of patches further comprises: 

fitting a plane to each patch of said selected plurality of the patches; 
obtaining a normal vector to each plane, 

said plurality of classes comprise one class representing the patches that are likely to 
represent a potential threat, another class representing the patches that possibly represent a 
potential threat, and another class representing the patches that are unlikely to represent a 
potential threat, wherein said classifying is based on the obtained normal vector for each patch 
and on 3D positions of each patch. 

9. (Original) The method of claim 8 further including the step of grouping patches that are 
likely to represent a potential threat together. 

10. (Original) The method of claim 9 fiuther including the step of creating a bounding box that 
represents a potential threat, wherein the bounding box is created in accord with the grouping of 
patches. 

1 1 . (Previously Presented) The method of claim 8 wherein detecting a potential threat in the 
tessellated depth map includes the steps of moving each patch after local tessellation to find the 
region of maximum stereo density near the original patch location, of discarding a patch if the 
region of maximum stereo density does not meet a predetermined criterion, and adding the patch 
to the plurality of the patches if the patch density meets the predetermined criterion. 
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12. (Original) The method of claim 8 wherein obtaining a normal vector includes the steps of 
calculating the third Eigen-vector of a matrix of patch values using a singular valued 
decomposition of the matrix, and then estimating the normal vector as the third Eigen-vector. 

13. (Previously Presented) A collision detection system, comprising: 

a stereo camera pair for providing imagery of a scene proximate a platform; 
a stereo image preprocessor for preprocessing said imagery, 

a depth map generator for producing a depth map from said preprocessed imagery; and 
a collision detector for tessellating the depth map into a number of patches, selecting a 
pluraUty of the patches of the depth map for processing, wherein said processing comprise 
classifying the selected plurality of patches of the depth map into a plurality of classes; detecting 
a potential threat in said tessellated depth map during the processing of the selected plurality of 
the patches, 

wherein said collision detector estimates size, position, and velocity of said detected 
potential threat; 

wherein said collision detector performs a trajectory analysis of said detected potential 

threat using said estimated position and said estimated velocity; 

wherein said coUision detector predicts a collision based on said trajectory analysis; and 
wherein said collision detector determines if a collision is imminent based on said 

collision prediction and on said estimated size. 

14. (Original) The system of claim 13, wherein said colUsion detector includes a filter for 
filtering image noise and outliers fi:om said estimated position and fi-om said estimated velocity 
before performing trajectory analysis. 

15. (Previously Presented) The system of claim 13 wherein said collision detector estimates 
said by identifying 2-dimensional feature correspondences from imagery produced in different 
time fi-ames and then obtains 3D correspondences of said 2-dimensional feature correspondences 
using said depth map. 
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16. (Original) The system of claim 13, fiirther including a host vehicle, wherein said stereo 
camera pair is mounted in fixed locations relative to said host vehicle. 

17. (Previously Presented) The system of claim 13 wherein said collision detector detects the 
potential threat in the tessellated depth map by: 

fitting a plane to the selected plurality of said patches; 
obtaining normal vectors to said selected plurality of patches, 

wherein said classifying is based on said normal vector for that patch and on 3D positions 
of each patch, as likely to represent a potential threat, as possibly representing a potential threat, 
or as being unhkely to represent a potential threat; and 

grouping patches that are likely to represent a potential threat together; and forming a 
bounding box around said potential threat based on said patch groupings. 

18. (Previously Presented) The system of claim 17 wherein said colHsion detector searches each 
patch after tessellation to find a densest part of said patch, discards said patch if said patch 
density does not meet a predetermined criterion, and adds said patch to said plurality of patches 
if said patch density meets said predetermined criterion. 

19. (Original) The system of claim 17 wherein obtaining normal vectors includes the steps of 
calculating for each patch a third Eigen-vector of a matrix of patch values using a singular valued 
decomposition of said matrix, and then estimating said normal vector as said third Eigen-vector. 

20. (Previously Presented) A computer readable medium having stored thereon a plurality of 
instructions, the plurality of instruction including instructions which, when executed by a 
processor causes the processor to perform the steps comprising: 

capturing and preprocessing an imagery of a scene proximate a platform; 
producing from the imagery a depth m^; 

tessellating the depth map into a number of patches and selecting a plurality of the 
patches of the depth map for processing, wherein said processing comprise classifying the 
selected plurality of patches of the depth map into a plurality of classes; 
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detecting a potential threat in the tessellated depth map during the processing of the 
selected plurality of the patches; 

estimating the size of the detected potential threat; 
estimating the position of the detected potential threat; 

estimating the velocity of the detected potential threat; 

performing a trajectory analysis of the detected potential threat using the estimated 
position and the estimated velocity; and 

performing a collision prediction based on the trajectory analysis. 

21. (Original) The computer readable medium of claim 20 that further causes the processor to 
filter the estimated position and the estimated velocity before performing trajectory analysis. 

22. (Original) The computer readable medium of claim 20 that further causes the processor to 
determine the velocity by identifying 2-dimensional feature correspondences &om imagery 
produced in different time frames. 

23. (Previously Presented) The computer readable medium of claim 22 that further causes the 
processor to determine velocity by obtaining 3D correspondences from the 2-dimensional feature 
correspondences and from the depth map. 

24. (Previously Presented) The computer readable medium of claim 20 that further causes the 
processor to detect the potential threat by the steps of: 

fitting a plane to each patch of the selected plurality of patches; 

obtaining a normal vector to each plane, 

wherein said classifying is based on said normal vector for that patch and on 3D positions 
of each patch, as likely to represent a potential threat, as possibly representing a potential threat, 
or as being unlikely to represent a potential threat. 

25. (Original) The computer readable medium of claim 24 that fiirther controls a computer to 
group patches together that are likely to represent said potential threat. 
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26. (Original) The computer readable medium of claim 25 that fiirther causes the processor to 
create a bounding box that represents the potential threat, wherein the bounding box is created in 
accord with the patch groupings. 

27. (Previously Presented) The computer readable medium of claim 20 that further causes the 
processor to detect a potential threat in said tessellated depth map by the steps of moving each 
patch after local tessellation to find the region of maximum stereo density near the original patch 
location, discarding said patch if the region of maximum stereo density does not meet a 
predetermined criterion, and adding said patch to said plurality of patches if said patch density 
meets said predetermined criterion. 

28. (Original) The computer readable medium of claim 20 that fiirther causes the processor to 
obtain normal vectors by the steps of calculating for each patch a third Eigen-vector of a matrix 
of patch values using a singular valued decomposition of said matrix, and then estimating said 
normal vector as said third Eigen-vector. 
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IX. EVIDENCE APPENDIX 

Awe Franke et al. Autonomous Driving Goes Downtown , LE.E.E. Intelligent Systems, 
pages: 40-48 1998 {Franke) cited by Examiner in office actions mailed on 02/28/2005, 09/08/05, 
05/18/06, 11/22/06, 03/08/07, 07/09/07 and on 12/17/07. 

Ming Yang et al. Vision-based Real-time Obstacles Detection and Tracking for 
Autonomous Vehicle Guidance . Real-time Imaging VI, Proceedings of SPIE Vol. 4666, pages 
65-74, 2002 {Yang) cited by Examiner in office actions mailed on 02/28/2005, 09/08/05, 
05/18/06, 11/22/06, 03/08/07, 07/09/07 and on 12/17/07. 
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X. RELATED PROCEEDINGS APPENDIX 

None. 
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